Part 1: Understanding big data in agriculture
Marie-Hélène Burle
Simon Fraser University’s Big Data Hub &
BC Centre for Agritech Innovation
November 20, 2024
SFU hosts the Cedar supercomputer—a cluster of 100,400 CPUs and 1,352 GPUs soon to be replaced by an even larger computer cluster
SFU also works with the Digital Research Alliance of Canada to offer researchers large amounts of computing power to solve challenging data and technology problems, as well as training to optimize their solutions
Since 2016, Simon Fraser University’s Big Data Hub has been offering workshops, events, and consulting services to researchers and industry partners helping them remain at the top of the fast evolving data landscape
Since 2022, SFU BCCAI has been helping small and medium enterprises in the farming industry to embrace technology driven solutions
Today
A (hopefully) friendly lecture to:
Tomorrow at 11am in the Mount Baker Room
An interactive workshop to:
Farmers were taking measurements (e.g. on soil moisture) manually creating low volumes of data
Farmers were taking measurements (e.g. on soil moisture) manually creating low volumes of data
Internet of Things (IoT) (e.g. thousands of soil moisture sensors) collects large volumes of data
There was a limited set of data a producer could collect
There was a limited set of data a producer could collect
There are so many different types of data (e.g. satellite images, market data gathered from internet browsing…)
A farmer could only gather so much data, even with a lot of employees
A farmer could only gather so much data, even with a lot of employees
Data is generated in real time and accumulates at high speed
All this data is key to the development of artificial intelligence (AI)
so…
What is AI?
Very loosely, you can think of neural networks (the most powerful form of AI) as an attempt to create a computer model that mimics the brain
In traditional computing, a programmer writes code that gives a computer detailed instructions of what to do
These instructions are called a program
Some action
With neural networks, instead of writing a program, a programmer writes a model, then feeds it lots of data and the model changes little by little over time
The model “learns” thanks to this data
Simplilearn has a video explaining how neural networks work in 5 min
This learning is nothing magical: some numbers in the model get tweaked a tiny bit, with each new piece of data, to make the model a little bit better
To get a very good model at the end—one that can write human language like ChatGPT for instance—you need A LOT OF DATA
You want a program able to detect tomatoes in an image
As human, this is straightforward
Yet, for a traditional approach, this is truly impossible because there are too many factors (location of the tomato in the image, quality of the picture, colour of the tomato…)
By feeding a very large number of images with and without tomatoes to a neural network we can train it to recognize tomatoes in images that it has never seen
The idea is not new, but it is only recently that we have had enough computing power, internet connectivity, and storage capacity to implement it
Farmers had to make decisions as best they could based on their experience and their limited data
Farmers had to take decisions as best they could based on their experience and their limited data
Farmers can use powerful models to make informed decision in real time. This can be followed by the automation of some action (e.g. watering)
To go a bit further than the video mentioned earlier, 3Blue1Brown by Grant Sanderson has a series of 4 videos on neural networks which is easy to watch, fun, and does an excellent job at introducing the functioning of a simple neural network
Open-access preprints:
Arxiv Sanity Preserver by Andrej Karpathy
ML papers in the computer science category on arXiv
ML papers in the stats category on arXiv
Distill ML research online journal
Please give us feedback by scanning the QR code:
Join us tomorrow at 11am in the Mount Baker Room for our 2nd session
We will have an interactive workshop to:
If you are unable to attend, you will find the slides here tomorrow, but it will be an interactive clinic with most of the material covered in the activity